Early computer networks consisted of a small number of devices attached together using a single cable. Computer networks have since evolved, however, beyond a simple collection of attached devices. Current computer networks may connect thousands of devices spread across large local areas, and these local area networks may in turn be connected together to form still larger networks such as, for example, the Internet.
Today's computer networks often interconnect widely-disparate devices distributed throughout multiple local networks into a single virtual network. Virtual networking technology allows devices located on a single physical network to operate as part of multiple virtual networks. Such virtual networks provide flexibility not available in early computer networks and allow network administrators to create layers of abstraction to simplify complex network topologies. For example, using a virtual network, an enterprise may have a virtual blade server chassis with routers spread across multiple physical locations that allow a data center which is split across multiple sites to act as if the data center is at a single site.
The predominate standard used to construct and access today's computer networks is Ethernet. Ethernet is a family of frame-based computer networking technologies for local area networks. Ethernet is promulgated by Institute of Electrical and Electronics Engineers (IEEE) in various standards specifications as part of the IEEE 802 family of standards. Ethernet defines a number of wiring and signaling standards for the Physical Layer of the Open Systems Interconnection (OSI) Networking Model, the means for network access at the Media Access Control (MAC) and Data Link Layer, and a common addressing format. At the physical layer, Ethernet networks are ubiquitous, carrying all kinds of traffic over multiple types of physical connections (wired or wireless), including 10 Mbps, 100 Mbps, 1 Gbps, 10 Gbps, 100 Gbps connections. The Ethernet service layer, generally referred to as Layer-2 because it is the MAC and Data Link Layer in the OSI networking model, provides the services generally required by a network. These network services typically include filtering, replication, forwarding broadcast, unicast, and multicast (BUM) traffic, and following a serviced topology, which may include virtual local area networks (VLANs), ATM segments of VLANs based on the ATM Lane Standard (ELANs), Ethernet Private Lines (ELINEs), and rooted multipoint Ethernet virtual connections (ETREEs).
VLAN services are specified in the IEEE 802.1Q standard and allow enterprise customers to configure various computing devices to communicate as if those devices were attached to the same broadcast domain, regardless of their physical locations. VLANs provide segmentation services traditionally provided by routers in local area network (LAN) configurations and address issues such as scalability, security, and network management. Bridges in VLAN topologies enforce the integrity of VLAN broadcast domains because such bridges are not permitted to bridge network traffic between VLANs. In this way, VLANs may provide broadcast filtering, security, address summarization, and traffic flow management. Network administrators may use VLANs to create multiple Layer 3 networks on the same Layer-2 bridge. For example if a Dynamic Host Configuration Protocol (DHCP) server, which broadcasts its presence, were plugged into a bridge, the DHCP server would serve any host device connected to the bridge. By using VLANs, however, a network administrator may easily split up the devices on the network so that some hosts will not use that DHCP server and will default to link-local addresses.
Because enterprise customers often have multiple networks distributed across multiple physical sites, customer's typically connected these physically separate networks together through the network of a network provider. For example, a company may connect its network at site A to its network at site B through a network provided by a telecommunications company. Despite the customer networks being connected through a provider network, devices on the different customer networks may still use VLAN services to communicate through the provider's network as though the devices were all located on the same LAN.
For the same reasons that enterprise customers take advantage of the VLAN services described in the IEEE 802.1Q specification, network providers also take advantage of VLAN services to provide flexibility, security, traffic flow management, and reduce their network administration burden. The drawback, however, is that under the IEEE 802.1Q specification, there are only 4096 identifiers available to specify different VLANs. Thus, a network provider and all the customers that provider serves must share the 4096 VLAN identifiers.
Because industry participants deemed such a limited number of VLAN identifiers inadequate to meet the needs of both customers and providers, the IEEE 802.1Q standard was amended by the IEEE 802.ad standard, often referred to as “Q-in-Q” or “stacked VLANs.” The IEEE 802.ad standard sets forth an architecture and bridge protocol to provide separate instances of MAC network services to multiple independent users of a provider network in a manner that does not require cooperation among the customers, and requires a minimum of cooperation between the customers and the provider of the MAC network service. Q-in-Q provides customers with the ability to configure their own VLANs inside the VLAN provided to the customer by a service provider. In such a manner, the service provider may configure one VLAN for the customer, and the customer can utilize that service provider VLAN to establish multiple customer VLANs.
In a manner similar to enterprise customers, network service providers often maintain multiple provider network domains, which are bridged together using a provider backbone bridging network. IEEE promulgates specifications for such a provider backbone bridging network in the IEEE 802.1ah standard. IEEE 802.1ah compliant networks provide complete separation of customer and service provider domains by encapsulating Ethernet frames with a service provider MAC header. Because the Ethernet frames are originally encapsulated in the customer network with a customer MAC header, this subsequent encapsulation with a service provider MAC header is often referred to as ‘MAC-in-MAC encapsulation.’ Using MAC-in-MAC encapsulation, Ethernet frames being sent in a customer's network from one domain to another through multiple service provider network domains contain two MAC headers. The customer MAC header provides routing information to the bridges in the customer's networks, while the service provider MAC header provides routing information to the bridges in the service provider's backbone bridging network.
To provide an overview of Ethernet header stacking of encapsulation, FIG. 1 sets forth a line drawing that illustrates exemplary Ethernet frame structures compliant with the IEEE 802.1 family of standards. FIG. 1 illustrates a traditional Ethernet frame 100 implemented in a customer's network according to the IEEE 802.1D standard. The customer frame 100 consists of a payload 101, a header type (EthType) 102 indicating that frame 100 is an 802.1D frame, a customer network source MAC address (C-SA) 103, and a customer network destination MAC address (C-DA) 104. The customer network source MAC address 103 specifies the source node in the customer network that originates the frame 100, while the customer network destination MAC address 104 specifies the destination node in the customer network to which the frame is bound for delivery.
As mentioned above, a customer may organize the nodes into various VLANs to provide traffic flow management, security, ease network administration, and the like. VLANs established by a customer for use within the customer's networks are generally referred to a ‘customer VLANs.’ In a network using customer VLANs, frame 100 is encapsulated as frame 110 to include a customer VLAN identifier (C-VID) 115 and a new header type (EthType) 116, indicating that the frame 110 is an 802.1Q frame. As used in this application, encapsulation may allow additional fields to be placed in any position relative to the encapsulated object, including interior to the original object, and does not require the additional fields be placed surrounding or at either end of the encapsulated object.
In a provider bridge (PB) network that bridges two customer networks, the frame 110 is further encapsulated as shown by frame 120, adding new fields for: a service VLAN identifier (S-VID) 127, and a new header type (EthType) 128 indicating the frame 120 is IEEE 802.1ad compliant. In a provider backbone bridging (PBB) network that bridges multiple PB networks, the frame 120 is further encapsulated to add additional fields for: a service identifier (I-SID) 131, a new header type (EthType) 132 corresponding to the service identifier 131, a backbone VLAN identifier 133, an additional header type (EthType) 134 indicating that the frame 130 is IEEE 802.1ah compliant, a backbone source MAC address (B-SA) 135 specifying the bridge through which the frame ingresses into the PBB network, and a backbone destination MAC address (B-DA) 136 specifying the bridge through which the frame egresses the PBB network.
For further explanation of header stacking or encapsulation in a IEEE 802.1ad provider bridge network, FIGS. 2 and 3 set forth network diagrams that illustrate an exemplary provider bridge network 200 interconnecting exemplary networks for customers A and B (201 and 202, respectively). In FIGS. 2 and 3, customer A 201 maintains three networks 203, 205, 207, while customer B 202 maintains two networks 204, 206. The provider bridge network 200 consists of six bridges, four provider edge bridges (PEB) 1-4 and two provider core bridges (PCB) 1-2. An edge bridge is a bridge through which frames ingress and egress the network 200—that is, an edge bridge is positioned at the ‘edge’ of the network topology. A core bridge is a bridge used to interconnect one or more edge bridges.
FIG. 2 illustrates a frame 220 at several stages as the frame 220 traverses the networks of FIG. 2 from customer equipment (CE)-11 210 in network 203 of customer A 201 to CE-31 212 in network 205 of customer A 201. In FIG. 2, the communications between CE-11 210 and CE-31 212 are implemented using a customer VLAN, and so the frame 220a from CE-11 210 is encapsulated with a customer VLAN header 230 that includes a customer VLAN identifier (C-VID) and a header type (EthType) specifying that the frame 220a is an IEEE 802.1Q compliant frame. The frame 220 includes a source MAC address (CE-11-SA) for CE-11 210, which indicates that CE-11 210 originated the frame (220a), and a destination MAC address (CE-31-DA) for CE-31 212, which indicates that the frame 220 is destined for CE-31 212.
When provider edge bridge (PEB)-1 240 receives the frame 220a, PEB-1 240 encapsulates the frame 220a into an 802.1ad frame 220b by adding a service VLAN header 231 to the frame 220. The service VLAN header 231 includes a service VLAN identifier (S-VID-A) assigned by the provider to customer A 201 and a header type (EthType) specifying that the frame is IEEE 802.1ad compliant. Using the service VLAN identifier to identify devices in the networks 203, 205, 207 of customer A 201, the provider bridges learn information about the MAC addresses of the computing devices in customer A's networks. From the learned MAC information, the provider bridges route the frame 220 through the provider network 200 from the PEB-1 240 through which the frame 220 ingresses the network 200 to the PEB-3 242 through which the frame 220 egresses the network 200. PEB-3 242 then de-encapsulates the frame 220b by removing to service VLAN header 231, leaving IEEE 802.1Q compliant frame 220a for delivery to CE-31 212.
Similarly, in FIG. 3, computing device CE-15 310 in network 204 of customer B 202 sends an IEEE 802.1Q compliant frame 320a to device CE-25 312 customer B's network 206. At PEB-1 240, frame 320a is encapsulated with a service VLAN header 331. The service VLAN header 331 includes a service VLAN identifier (S-VID-B) assigned by the service provider to customer B 202 and a header type (EthType) specifying that the frame is IEEE 802.1ad compliant. The ingress bridge 240 of the provider network 200 forwards the frame 320b to the egress bridge 342 of the provider network 200, which in turn de-encapsulates the frame 320b by removing the service VLAN header 331, leaving IEEE 802.1Q compliant frame 320a for delivery to CE-35 312. As the provider bridges receive various frames on their ports, the bridges learn the MAC addresses of the devices in customer B network by monitoring the MAC addresses associated with each service VLAN identifier assigned to customer B 202 by the network provider.
From the description above, readers will note that in a provider bridge network, the service provider uses one or more SVLANs to transport frames for a customer's VLANs between multiple customer networks. To determine the forwarding path for each service VLAN through the provider's bridge network, the provider bridges often use the Generic VLAN registration protocol (GVRP) or the Multiple VLAN Registration Protocol (MVRP). For multicast traffic containment, provider bridges may use the Generic Attribute Registration Protocol Multicast Registration Protocol (GMRP) or the Multiple Multicast Registration Protocol (MMRP). For purposes of forwarding traffic, provider edge bridges learn all customer equipment MAC addresses and forward customer frames based on the service VLAN identifier and customer VLAN identifier pairs, while provider core bridges learn all customer equipment MAC addresses, but forward customer frames based only on the service VLAN identifiers. Within a particular provider bridge network, a given customer equipment MAC address is at the same site for all service VLANs.
Even with the stacked VLAN protocols, provider bridge networks have scaling problems. Because only 4,096 service VLAN identifiers are available under current protocols, provider bridge networks are limited in the number of customer networks they can serve effectively and efficiently. Further, because provider bridges learn the MAC addresses for all customer devices, scaling issues often arise when a provider serves one or more customers with large virtual networks. In addition, there is a potential for interaction between customer and service provider control protocols.
Provider Backbone Bridge (PBB) networks are one attempt to alleviate these issues because PBB networks allow a service provider to partition a large provider bridge network into several smaller provider bridge networks that are interconnected by the PBB network. For further explanation, FIG. 4 sets forth a network diagram that illustrates an exemplary provider backbone bridge network 400 interconnecting exemplary provider bridge networks 410, 420, 430. The PBB network 400 of FIG. 4 consists of four provider backbone edge bridges (BEB)—that is, BEB-11 416, BEB-12 418, BEB-21 434, and BEB-22 436—and two provider backbone core bridges (BCB)—that is, BCB-1 401 and BCB-2 402.
FIG. 4 illustrates a frame 412 at several stages as the frame 412 traverses the networks of FIG. 4 from customer equipment (CE)-11 411 to CE-34 431. The customer utilizing the PB networks 410, 430 and the PBB network 400 has grouped CE-11 411 and CE-34 431 in the same customer VLAN. Thus, frame 412a from CE-11 411 includes a customer VLAN header 415. The customer VLAN header 415 includes the customer VLAN identifier (C-VID) assigned by the customer for the customer VLAN and a header type (EthType) that specifies that the frame 412a is IEEE 802.1Q compliant.
When the frame 412a reaches the provider edge bridge (PEB)-11 413, PEB-11 413 encapsulates the frame with a service VLAN header 417. The service VLAN header 417 includes the service VLAN identifier (S-VID) assigned to the customer by the network provider and a header type (EthType) that specifies that the frame 412a is IEEE 802.1ad compliant.
When the frame 412b reaches the provider backbone edge bridge (BEB)-11 416, BEB-11 416 encapsulates the frame with a backbone header 419. The backbone header 419 includes a service identifier (I-SID), a new header type (EthType) corresponding to the service identifier, a backbone VLAN identifier, an additional header type (EthType) indicating that the frame 412c is IEEE 802.1ah compliant, a backbone source MAC address (BEB-11-SA) specifying the backbone edge bridge through which the frame ingresses into the PBB network 400, and a backbone destination MAC address (BEB-22-DA) specifying the backbone edge bridge 436 through which the frame egresses the PBB network 400.
The frame 412c is routed through the provider backbone bridge network 400 from the ingress bridge, BEB-11 416, to the egress bridge, BEB-22 436. BEB-22 436 de-encapsulates the frame 412c by removing the backbone header 419, leaving the frame 412 IEEE 802.1ad compliant. BEB-22 436 then sends the frame 412 along to PEB-31 433 in the IEEE 802.1ad compliant provider bridge network 430. PEB-31 433 further de-encapsulates the frame 412 by removing the service VLAN header 417, leaving the frame 412 IEEE 802.1Q compliant. PEB-31 433 then forwards the frame 412 along to CE-34 431 for delivery.
In a provider backbone bridge network, readers will note that there is clear demarcation between customer and service provider domains. MAC address learning for customer equipment is limited to the provider edge bridge, and the I-SID field allows separation of Ethernet as a service from Ethernet as infrastructure.
As mentioned above, the networking architectures described by the IEEE 802.1Q, 802.1ad, and 802.1ah standards allow enterprise customers to establish multiple networks that are geographically dispersed, yet operate as a single virtual network. These physically separate LANs communicate through PB and PBB networks using forwarding trees established using a spanning tree protocol. The spanning tree protocol is an OSI Layer-2 protocol that ensures a loop-free topology for any bridged LAN. This protocol allows a network design to include spare (redundant) links to provide automatic backup paths if an active link fails, without the danger of bridge loops, or the need for manual enabling or disabling of these backup links. Bridge loops must be avoided because such loops result in traffic that floods the network. The spanning tree protocol is defined in the IEEE 802.1D standard, and, as the name suggests, it creates a spanning tree within a mesh network of connected Layer-2 bridges, and disables those links that are not part of the tree, leaving a single active path between any two network nodes.
There are certain disadvantages to the spanning tree protocol used in the networks described above with reference to FIGS. 1-4. Because the spanning tree protocol disables links that are not part of the forwarding tree, bottlenecks are often created by concentrating traffic onto selected links. Also, due to the nature of the spanning tree protocol, temporary loops may develop if spanning tree messages are lost or as the network topology changes because nodes are brought on-line or taken off-line or moved in the network. During periods when temporary loops exists, frames may flood the network because the standard Ethernet header does not contain a time-to-live field or hop count that specifies when a frame has become stale and should be discarded. Further, the paths developed between nodes of the network are not necessarily the pair-wise shortest path, but rather are the paths that remain after the spanning tree protocol eliminates redundant paths.
In an attempt to create a network solution without the disadvantages of the spanning tree protocol, the Internet Engineering Task Force (IETF) has developed additional protocols. One such protocol is the “Transparent Interconnection of Lots of Links” (TRILL) protocol. The TRILL protocol and other similar specifications require the use of router-bridges (R-Bridges) to route frames hop-by-hop through the network along the shortest path to the frames destination in the network, e.g., the network's edge bridge. For further explanation, FIG. 5 sets forth a network diagram that illustrates an exemplary TRILL network 500, also known as a router-bridge network, because the devices behave as both a router and a bridge at the ISO Layer-2 level. The TRILL network 500 includes three core router-bridges (CRB)—that is, CRB-4 540, CRB-5 542, and CRB-6 544—as well as five edge router-bridges (ERB)—that is, ERB-1 530, ERB-2 532, ERB-8 534, ERB-9 536, and ERB-7 538. The TRILL network 500 is provided to customers by a network provider. The edges of the TRILL network 500 typically extend into the datacenter where the customer equipment is housed. In fact, often each equipment rack includes a bridge that operates as an edge of the TRILL network 500. These TRILL networks can extend throughout one or more datacenters to interconnect various networks.
The TRILL network 500 of FIG. 5 interconnects two IEEE 802.1Q networks 510, 520. 802.1Q network 510 is connected to the TRILL network 500 through ERB-1 530. 802.1Q network 520 is connected to the TRILL network 500 through ERB-8 534 and ERB-9 536. 802.1Q network 510 includes a legacy bridge (LB)-55 514 and a LB-56 515, neither of which support MAC-in-MAC encapsulation or are TRILL compliant. 802.1Q network 520 includes EB-85 524 and EB-96 526. End point devices include customer equipment (CE)-11 512 connected to LB-55 514, CE-31 522 connected to EB-85 524, and CE-77 539 connected to ERB-7 538. End point devices CE-11 512, CE-31 522, and CE-77 539 can be any type of computing device, including workstations, servers, network devices, and so on.
FIG. 5 illustrates a frame 518 at several stages as the frame 518 traverses the networks of FIG. 5 from CE-11 512 to CE-31 522. The customer utilizing the TRILL network 500 to bridge multiple 802.1Q networks has grouped CE-11 512 and CE-31 522 in the same customer VLAN. Thus, frame 518a sent by LB-55 514 for CE-11 512 includes a customer VLAN header 519. The customer VLAN header 519 includes the customer VLAN identifier (C-VID) assigned by the customer for the customer VLAN and a header type (EthType) that specifies that the frame 518a is IEEE 802.1Q compliant.
When the frame 518a reaches the ERB-1 530, ERB-1 530 encapsulates the frame 518 with a TRILL header 551. The TRILL header 551 includes a TRILL source nickname (ERB-1-SN) specifying ERB-1 as the ingress edge router-bridge for the frame 518, a TRILL destination nickname (ERB-8-DN) specifying ERB-8 534 as the egress edge router-bridge for the frame 518, various TRILL flags, a hop count, and a header type (EthType) indicating that frame 518b is a TRILL frame. TRILL nicknames are assigned to every router-bridge in a TRILL network using a dynamic nickname acquisition protocol or other protocols as will occur to those of skill in the art.
ERB-1 530 also encapsulates the frame 518 with an Ethernet MAC header 552. The MAC header 552 includes a outer transport VLAN identifier (OT-VLAN-ID), a header type (EthType), a source MAC address (ERB-1-SA) specifying ERB-1 530 as the node transmitting the frame 518b on the next network hop through the network 500, and a destination MAC address (CRB-5-DA) specifying CRB-5 542 as the node receiving the frame 518b on the next network hop through the network 500. ERB-1 530 then sends the frame 518 to CRB-5 542, which routes the frame through the TRILL network 500 to CRB-4 540 based on a shortest path to ERB-8 534. As the frame traverses the TRILL network 500, the MAC header 552 is changed at each hop to update the source and destination MAC addresses for the next network hop. Accordingly, when the frame 518c passes from CRB-4 540 to ERB-8 534, the frame 518 includes MAC header 562. The MAC header 562 of FIG. 5 includes a source MAC address (CRB-4-SA) specifying CRB-4 540 as the node transmitting the frame 518c on the next network hop through the network 500 and a destination MAC address (ERB-8-DA) specifying ERB-8 534 as the node receiving the frame 518c on the next network hop through the network 500. Upon receiving the frame 518c, ERB-8 534 de-encapsulates the frame 518 by removing the MAC header 562 and the TRILL header 551, leaving frame 518a for delivery to CE-31 522 through EB-85 524.
Readers will note that the TRILL network 500 operates as a massive switch fabric from the perspective of the customer network. Frames enter the TRILL network at an ingress bridge and are routed along the shortest path hop-by-hop through the TRILL network to an egress bridge for delivery to a customer network. As the bridges in the TRILL network learn the MAC address of the customer equipment in the customer networks to which the TRILL bridges connect, the TRILL bridges share information among one another about which customer MAC addresses are related to which TRILL bridge nickname. ISO Layer-2 frames with known unicast addresses are routed hop-by-hop based on TRILL nicknames of the ingress and egress edge router-bridges by the transit or core router-bridges. ISO Layer-2 multicast traffic can also be routed hop-by-hop based on multicast distribution trees. In addition to TRILL, there are other provider bridging protocols such as Shortest Path Provider Backbone Bridging (SPPBB) described in IEEE 802.1aq that represent attempts by industry participants to utilize the shortest path through the network to forward frames between edge bridges.
Regardless of whether the TRILL protocol or the SPPBB protocol is used to implement a particular shortest path network, both protocols require that network bridges support MAC-in-MAC encapsulation. These protocols use the additional Ethernet header—that is, Ethernet header 552 of FIG. 5—to route the frame within a network along the shortest path from the bridge through which the frame ingressed the network to the bridge through which the frames egresses the network.
Legacy bridges, however, do not support MAC-in-MAC encapsulation and cannot properly form frames for use in shortest path networks such as TRILL network 500. In order to carry traffic from and to the legacy bridges 514 and 515 across the TRILL network 500, edge bridge 530 operates as an appointed-forwarder that interacts with the other bridges of the TRILL network 500 for the VLANs on network 510. The drawback to this arrangement, however, is that edge bridge 530 becomes a single point of failure for all traffic to and from the 802.1Q network 510 when operating as an appointed-forwarder for those VLANs. The edge bridge 530 may also becomes a bottleneck for network traffic between the legacy bridges in the 802.1Q network 510 and the TRILL network 500. Thus, such a configuration decrease overall reliability and efficiency.